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\  1.  INTRODUCTION. 

There  are  four  basic  questions  related  to  efficiency  and  capability 
which  are  of  particular  interest  to  officials  in  the  military  services 
who  are  interested  in  better  ways  of  evaluating  military  capability  and 
efficiency:  (i)  What  level  of  military  capability  can  the  St. vices  achieve 
with  available  resources?  (ii)  What  capability  is  required,  and  where  are 
the  shortfalls?  (iii)  What  resource  acquisitions  or  redistributions  are 
needed  to  achieve  maximum  improvement  in  efficiency  and  effectiveness?  and 
(iv)  How  can  management  systems  be  changed  to  improve  the  identification  and 
correction  of  factors  which  limit  the  readiness  and  efficiency  of  our 
military  operations? 

The  last  question,  which  differs  in  its  emphasis  from  the  other 
three,  provides  an  opening  to  the  topics  that  will  be  addressed  in  this 
report.  In  particular, -we  yhaTT-reportHrfTe  results  from  a  study  of  DEA 
(Data  Envelopment  Analysis)  as  a  method  for  evaluating  the  efficiency  of 
Air  Force  WingS'-or,  more  precisely,  their  maintenance  operations--as 
elements  in  Numbered  Units  in  the  U.  S.  Air  Force,  _ 

By  way  of  background,  DEA  is  an  approach  to  evaluating  not-for-profit 
entities  which  has  emerged  from  research  by  A.  Charnes,  W.  W.  Cooper  and 
colleagues.  In  addition  to  the  initial  formulations,  these  persons  have  been 
responsible  for  a  series  of  studies  directed  to  testing  DEA  for  use  in 
measuring  the  efficiency  of  a  wide  variety  of  not-for-profit  institutions 
ranging  from  schools  and  hospitals  to  ar.ny  recruitment  districts.-'^  In  such 
applications,  we  may  note,  DEA  has  advantages  like  the  following:  (1)  It 


y  See  Rhodes  [  14  ],  Sherman  [  15  ]  and  Charnes,  Cooper,  Divine,  Klopp  and 
Stutz  [  7  ]  as  well  as  [  8  ]  and  [  10  ]. 


deals  directly  with  multiple  outputs  and  multiple  inputs  and  yields  an 
overall  measure  of  relative  efficiency  for  each  pertinent  decision  making 
unit  without  requiring  either  (a)  an  a  priori  selection  of  weights  or 
(b)  explicitly  hypothesized  forms  of  relations  between  the  various  inputs 
and  outputs.  (2)  Possible  sources  of  inefficiency  in  each  decision  making 
unit  are  pinpointed  for  further  study.  Finally,  (3)  tradeoff  ratios  are 
provided  for  still  further  improvement  even  after  efficiency  has  been 
attained. 

The  focus  of  this  report  will  be  on  point  (1),  with  points  (2)  and  (3) 
being  dealt  with  only  sketchily.  The  DMUs  (  =  Decision  Making  Units)  selected 
for  this  study  are  14  Air  Force  Wings  which  are  elements  of  two 
Numbered  Air  Forces.  Their  efficiency  will  be  determine  by  reference 
to  output  and  input  variables  which  are  commonly  used  in  gauging  the 
performance  of  aircraft  maintenance  units  in  these  operations. 

Formally,  100%  efficiency  is  attained  for  any  DMU  only  when 

(a)  None  of  its  outputs  can  be  increased  without  either 

(i)  Increasing  one  or  more  of  its  inputs  or 
(ii)  Decreasing  some  of  its  other  outputs 

(b)  None  of  its  inputs  can  be  decreased  without  either 

(i)  Decreasing  some  of  its  outputs  or 
(ii)  Increasing  some  of  its  other  inputs. 

Via  this  definition  we  avoid  the  need  for  assigning  a  priori  measures  of 
relative  importance  to  the  different  outputs  and  inputs.  Hence  output  or 
input  inefficiencies,  when  identified  in  any  DMU,  may  then  be  corrected 
without  worsening  any  other  input  or  output.  Moreover,  we  are  not  con¬ 
fined  to  indexes  for  ranking  or  ordering  of  inefficient  DMUs.  In 


particular,  we  will  be  able  to  estimate  amounts  of  inefficiency  in  each 


such  DMU  and  we  shall  be  able  to  identify  where  the  inefficiencies  are 
located  (i.  e.,  in  which  inputs  or  outputs).  How  these  amounts  are 
determined  by  means  of  DEA  models  and  methods  of  analysis  and  interpretation 
will  be  described  in  the  sections  that  follow. 

The  above  definition  is  formulated  so  that  efficiency  may  be  determined 
relative  to  prior  theoretical  knowledge  or  on  the  basis  of  some  predeter¬ 
mined  norm  from  studies  such  as  are  available  in  physics  and  engineering, 
or  like  disciplines  and  endeavors.  For  the  kinds  of  situations  which 
are  of  interest  in  this  report,  however,  such  prior  knowledge  will 
generally  not  be  available.  Hence  we  extend  the  above  definition  to  one 
which  involves  only  relative  efficiency  for  use  on  the  kind  of  data  that 
are  likely  to  be  available: 

100%  relative  efficiency  is  attained  by  any  DMU  only  when 
comparisons  with  other  relevant  DMUs  do  not  provide  evidence 
of  inefficiency  in  the  use  of  any  input  or  output. 

Via  this  characterization  the  preceding  definition  is  adjusted  for  immediate 

application  to  data  of  the  kind  we  shall  be  considering.  We  should  also  note, 

however,  that  other  combi r.-*t ions  of  the  above  definitions  are  also  possible 

so  that,  in  addition,  pertinent  aspects  of  any  theoretically  grounded  norms 

or  other  types  of  available  knowledge  may  also  be  used  in  common  with  other 

data  when  required. 

We  now  conclude  this  introductory  section  by  pointing  to  other  studies 
which  have  compared  DEA  to  other  approaches  such  as  ratio  and  regression 
analyses,  simulation  and  similar  approaches.  For  instance,  D.  Sherman  in 
[  15  ]  compared  DEA  with  ratio  and  regression  approaches  for  use  on  data 
generated  by  Massachusetts  hospitals.  In  this  comparison  DEA  performance  was 
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generally  favored  because  it  avoided  being  restricted  to  considering  only 
1  or  2  variables  at  a  time--perhaps  at  aggregated  level s--which  is  an 
inherent  limitation  of  ratio  approaches.  Similarlys  recourse  to  DEA  made 
it  possible  to  avoid  the  assumptions  of  known  functional  forms  between 
the  inputs  and  outputs— as  is  required  in  regression  approaches-- along 
with  the  difficulties  that  are  often  encountered  in  such  approaches  when 
dealing  with  multiple  variables  that  interact  in  other  than  theoretically 
prescribed  manners,  and  so  on.  Finally,  as  shown  in  [  10  ],  the 
optimizing  principles  used  in  regression  and  DEA  are  quite  different  and 
hence  may  be  expected  to  yield  results  which  will  favor  DEA  when  evalu¬ 
ation  of  individual  DMUs  is  desired.  Regression  approaches,  for  example, 
tend  to  optimize  by  reference  to  "average"  or  "representative"  behavior-- 
as  determined  via  a  least  squares  analysis--across  all  observations. 

DEA,  on  the  other  hand,  optimizes  on  each  observation  and  does  so  not  by 
reference  to  "average"  or  "representative"  behavior  but  by  reference  to 
subsets  of  "efficient"  DMUs.-'^ 

The  manner  in  which  this  is  accomplished  will  be  described  in  the 
sections  that  follow.  Here  we  are  only  referencing  the  fact  that  such 
comparisons  and  evaluations  have  been  made.  Finally,  therefore,  we  also 
note  that,  in  contrast  to  simulation  approaches,  DEA  does  involve 
optimizations  that  make  it  possible  to  identify  its  results  with  availabl 
bodies  of  knowledge  in  economics,  management  science  and  mathematics 

]_/  As  shown  in  [  1  ],  DEA  exhibits  similar  advantages  even  relative  to 
regressions  adjusted  to  obtain  frontier  estimates. 
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(which  are  based  on  similar  optimizing  principles).  Moreover,  as  will  be 

seen,  DEA  lends  itself  to  straightforward  uses  directly  on  the  data 

without  the  need  for  developing  numerous  intermediate  treatments  and 

relations,  as  in  most  simulation  modeling.  Finally,  computer  codes  for 

effecting  such  DEA  studies  are  now  freely  available  so  that  possible 

1  / 

trouble  from  this  quarter  is  also  avoided.-' 


?  .  THE  DEA  MODEL 


A  fuller  theoretical  development  of  DEA  is  provided  in  the  appendix 
to  this  report.  Here  we  therefore  record  only  the  following  version  of 
the  Charnes  Cooper  Rhodes  (CCR)-^  ratio  form  of  the  model  in  order  to 
explain  what  is  being  dona  in  the  sections  that  follow. 

Objective: 
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.,14-  DMUs(=  Air. Force  W'ngs) 
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Positivity  ,  0  <  u^  ;  r  =  1 , . . . ,s 
Constraints*  0  <  v.,-  ;  i  =  l,.,.,m 


Data: 

Outputs:  ypj  =  observed  amount  of  r^^  output  for  j^^  Air  Force  Wing 
Inputs:  =  observed  aiiount  of  Input  for  j^*^  Air  Force  Wing 


ly  A  code  as  deve'loped  by  A.  Ali  and  0.  Stutz  for  these  purposes  is  available 
from  the  Center  for  Cybernetic  Studies,  BEB  454C,  The  University  of  Texas  at 
Austin,  Austin,  Texas  78712 
^  See  [  8  ]  and  [  9  ]. 
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Any  one  of  the  DMUs  represented  in  the  constraints  may  be  selected 

for  such  an  evaluation.  The  selected  DMU  is  then  represented  in  the 

objective  as  well  as  in  the  constraints.  Hence  we  must  have 
* 

max.  ho  =  Hq  £  1  in  order  to  satisfy  the  constraints.  Within  this 

*  * 

limitation  the  optimal  u^-  =  u-j  and  optimal  v^  =  v^,  are  then  chosen 
so  that  the  DMU  being  rated,  is  accorded  the  highest  possible  ho 

value. 

All  of  the  observed  outputs,  y^.-,  and  all  of  the  observed  inputs, 
xij,  are  assumed  to  be  available  as  known  positive  constants.  That  is, 
each  of  the  j  =  1,  2,...,  n  DMUs  is  assumed  to  have  used  positive  amounts 
of  each  pertinent  input  and  to  have  produced  positive  amounts  of  each 
pertinent  output. 

Some  of  the  inputs  may  be  varied  at  the  discretion  of  a  manager  and 
some  may  not.  An  example  of  such  a  "non-discretionary"  input  is  pro¬ 
vided  by  the  weather g.,  in  the  form  of  flying  days  or  days  available 

* 

for  flying.  Nevertheless,  the  efficiency  rating  max  ho  =  Hq  should 
take  this  input  into  account  and,  moreover,  it  should  do  so  with  the 
mixes  of  inputs  and  outputs  (whatever  they  may  be)  which  are  pertinent 
to  the  performance  of  all  aircraft  wings  that  utilize  these  same  inputs 
and  produce  these  same  outputs.  Hence,  both  discretionary  and  non- 
discretionary  inputs  were  used  in  the  study  that  will  be  discussed  in 
this  paper,  and,  moreover,  a  mix  of  various  types  of  wings  (training  and 
operational)  as  well  as  various  types  of  aircraft  were  used  in  this 
study  in  order  to  test  possible  further  ranges  and  limits  of  DEA. 

ly  Further  special  attention  which  is  needed  to  treat  such  ncn-ciiscretionary 
(or  partially  non-discrationary)  inputs  is  not  discussed  in  this  report. 


As  shown  in  (1),  all  14  wings  are  represented  in  the  less-than-uni ty 
constraints.  As  will  later  be  seen,  however,  it  is  the  job  of  DEA  to 
single  out  a  "best"  comparison  set  (or  rather  subset)  for  determining  the 
efficiency  value  of  each  DMU  being  rated. 

This  topic  will  subsequently  be  developed  in  more  detail.  Here  we 
may  note  that  the  "positivity  constraints"  imply  that  each  output  and  each 
input  has  "some"  value  with,  as  already  noted,  specific  optimal  values 

*  if 

and  Vi  assigned  to  each  such  output  and  input.  Optimality  is  determined 
in  each  case  by  reference  to  the  DMU  being  evaluated,  which  is  represented  in 

if 

the  objective  as  well  as  the  constraints  a  maximum  =  h^  £  1  resul^ng. 

★  * 

These  optimal  Ur  and  have  a  variety  of  uses  in  their  own  right 
as  when,  for  instance,  they  are  employed  to  determine  further  tradeoff 
possibilities  after  efficiency  has  been  attained.  They  are  also  called 
"virtual  rates  of  transformation"  which  define  a  "virtual  output" 

s 

yo  =  Z  Up  y^o  (2.1) 

r  =  1 

and  a  "virtual  input" 

m 

Xq  =  Z  v-j  x-jQ  (2.2) 

i  =  1 

so  tfiat  also 

^0  =  V^o»  (2.3) 

wi  th 

ho*  =  yo*/^o  (2.4) 

when  an  optimum  is  achieved.  In  other  words,  our  definition  (1)  still  retai 
contact  with  the  classical  ratio  definitions  of  efficiency  in  engineering, 
physics  (and  other  fields),  while  also  accommodating  multiple  output  and 
multiple  input  situations. 
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3.  MAINTENANCE  PERFORMANCE  STUDY  DETAILS 

Air  Force  tactical  fighter  wings  are  expected  to  maintain  the  readiness 
of  their  assigned  aircrews,  tactical  fighter  aircraft  and  ground  support. 
This  is  a  complex  problem  and  the  following  sections  present  a  small 
numerical  example  which  can  help  to  provide  insight  into  the  complexity 
of  the  wing  maintenance  evaluation  proplem.  This  will  then  provide  a  basis 
for  illustrating  the  possible  uses  of  DEA  in  such  contexts.  Although 
obtained  from  actual  Air  Force  data  bases,  the  numbers  used  in  this  study 
have  been  treated  and  abbreviated  in  a  variety  of  ways  so  that  the  results 
reported  here  should  only  be  regarded  as  illustrative  in  character. 

The  input  and  output  measures  in  this  analysis  are  similar  to  those 
used  by  Air  Force  commanders  and  resource  managers.  Chosen  to  highlight 
key  objectives  and  operating  characteristics,  the  inputs  and  outputs 
selected  for  this  study  reflect,  either  directly  or  indirectly,  the 
following  peace-time  initiatives  of  wing  aircraft  maintenance  organizations 

Data  were  obtained  from  fourteen  actual  tactical  fighter  wings,  eight 
of  which  are  organized  under  o.ie  intermediate  headquarters  (I)  and  the 
remaining  six  under  another  (II).  Both  intermediate  headquarters  report 
to  the  Tactical  Air  Coimiand  (TAC)  Headquarters.  The  selected  wings  (A,  B,. 
N)  fall  into  one  of  three  mission  categories:  (1)  combat  operations, 

(2)  training,  or  (3)  both;  and  each  wing  has  one  assigned  aircraft  type. 

The  identity  of  the  wings  will  not  be  divulged  since  the  results  reported 
here  are  illustrative.  See  Table  2.  for  generic  classifications  of  wing  and 
aircraft  types. 


nNG  AIRCRAFT  MAINTENANCE  OPERATIONS 


SIMPLE:  EASY  TO  TROUBLESHOOT  AND  FIX 


The  four  outputs  and  eight  inputs  used  in  this  study,  with  accompany¬ 
ing  definitions  of  the  abbreviations  employed,  are  shown  in  Table  3.  Thus, 
TOTSORTFLO  refers  to  the  total  number  of  sorties  flown  while  MICAPHOURS 
refers  to  the  total  number  of  hours  in  which  the  aircraft  possessed  by 
each  wing  were  fully  or  partially  mission  capable.  It  should  be  notea  that 
the  first  output  is  measured  in  units  of  10  while  the  second  one  is  not 

similarly  adjusted.  This  is  done  for  computational  convenience,  as  may  be 

★ 

done  with  DEA,  since  the  optimum  h^  =  h^  does  not  vary  with  the  units  of 
measurement  employed.  MICAPHRS,  on  the  other  hand,  is  stated  in  the 
originally  reported  units. 

TNMCHM,  the  total  number  of  hours  in  which  aircraft  in  each  wing  were 
not  mission  capable,  is  treated  in  yet  another  manner.  It  is  measured  by 
subtracting  the  observed  values  from  100,000  hours.  That  is  for  this  variable 
we  use 

TNMCHM  =  100,000  -  TNMCHM 

so  that  TNMCHM  increases  as  TNMCHM,  the  observed  (  =  reported)  value, 
decreases  and  vice  versa.  Any  other  similarly  "large"  number  could  have 

been  used  in  place  of  100,000  hours.  The  idea  is  to  preserve  the  isotone 
property  for  this  output  (see  Appendix). 

This  is  the  approach  that  was  used  for  this  output  and  other  outputs  and 
inputs,  as  indicated  in  Table  3,  for  the  monthly  evaluations  that  are 
reported  in  Tables  6  and  7.  For  the  cumulative  evaluations  reported  in 
Tables  4  and  5,  however,  an  alternate  approach  was  used  in  which  these  output 

and  input  values  were  converted  to  reciprocals  and  the  results  stated  in 
8 

units  of  10  hours.  This  was  done  to  maintain  comparability  with  the 
magnitudes  of  other  inputs  and  outputs — and  in  order  to  reduce  the  possibility 
of  efficiency  values  being  affected  by  computational  (roundoff)  error--in  the 
data  used  for  the  cumulative  evaluations  reported  in  Tables  4  and  5. 
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Such  scalings  and/or  rescalings,  we  might  note,  are  admissible  in  a 

•k 

DEA  analysis  since  they  do  not  affect  the  resulting  h^  values,  This, 
too,  is  an  advantage  not  only  for  purposes  of  computation  but  also  for 
avoiding  the  need  for  recourse  to  the  elaborate  procedures  that  are 
sometimes  needed  to  avoid  the  effects  of  using  different  scales  in 
different  parts  of  the  same  study.  In  other  words,  the  properties  of 
DEA  modeling  are  such  that  the  different  scales  used  in  Tables  4  and  5 
and  in  Tables  6  and  7  do  not  alter  or  destroy  the  comparability  of 
the  resulting  efficiency  values. 

Similar  remarks  are  applicable  to  the  inputs  with  one  additional 
feature  that  might  be  observed  in  AIRMENMEAS.  As  noted  in  its  definition, 
this  measure  is  intended  to  accommodate  a  comparison  between  authorized 
or  planned  and  assigned  or  actual  inputs  of  airmen.  This  is  accommodated 
by  forming  the  ratio  of  the  latter  to  the  former.-''^ 


y  Such  ratio  approaches  can  cause  difficulties  that  need  to  be  guarded 
against  in  uses  of  DEA.  See  Sher-man  [15],  In  the  present  study 
these  ratios  were  used  so  that  the  resulting  solutions  could  also 
make  it  possible  for  higher  commands  to  ascertain  the  effects  of 
their  authorizations  on  wing  efficiencies. 
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TABLE  3 


INPUT/OUTPUT  VARIABLE  DEFINITION 


OUTPUTS 

INPUTS 

TOTSORTFLO 

TOTPOSSAIR 

MICAPHOURS 

TNMCHS 

TNMCHM 

OFASSIGNED 

FIXRATE 

OFMEASURE 

AIRMENASSI 

AIRMENMEAS 

NOCHARGLOS 

CANNRATE 

DEFINITIONS  OF  INPUT  AND  OUTPUT  VARIABLES 


TOTSORTFLO 

MICAPHOURS 

TNMCHM 

FIXRATE 

TOTPOSSAIR 

TNMCHS 

OFASSIGNED 

OFMEASURE 

AIRMENASSI 

AIRMENMEAS 

NOCHARGLOS 

CANNRATE 


Total  number  of  sorties  flown  by  each  wing  in  a  specific  month 
(x  10) 

Total  number  of  hours  in  a  specific  month  in  which  the  possessed 
aircraft  in  each  wing  were  fully  or  partially  mission  capable 
lOOjOOO  minus  total  number  of  hours  in  a  specific  month  in  which 
the  possessed  aircraft  in  each  wing  were  not  mission  capable  due 
to  maintenance  problems* 

The  percentage  of  code  3  breaks  in  a  wing  in  a  specific  month 
which  were  fixed  within  given  time  intervals 
The  average  number  of  aircraft  in  the  wing  in  a  specific  month 
(X  100) 

100,000  minus  total  number  of  hours  in  a  specific  month  in  which 
the  possessed  aircraft  in  each  wing  were  not  mission  capable  due 
to  supply  problems* 

Number  of  officers  assigned  to  each  wing  in  a  specific  month 
(x  1,000) 

Assigned  officers  divided  by  authorized  officers  in  a  wing 
(x  1,000) 

Number  of  airmen  assigned  to  each  wing  in  a  specific  month 
Assigned  airmen  divided  by  authorized  airmen  in  a  wing  (x  1,000) 
1,000  minus  number  of  sortie  losses  due  to  external  reasons  Dike 
weather,  higher  headquarters,  air  traffic  control,  etc.)  (x  10) 
100  minus  total  number  of  cannibalizations  in  a  wing  in  a 
specific  month  divided  by  the  number  of  sorties  flown  (x  1,000) 


*This  was  the  approach  used  for  the  monthly  evaluations  used  in  Tables  6  and  7. 
For  the  cumulative  runs  reported  in  Tables  4  and  5,  however,  an  alternate 
approach  was  used  in  which  reciprocals  of  the  reported  outputs  were  used  and 
restated  in  units  of  10°  hours. 
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4.  PRELIMINARY  RESULTS 

For  purposes  of  the  present  study,  attention  was  restricted  to 
already  available  data  for  selected  months  during  the  period  October  1981 
through  May  1982.  Manpower  data  were  obtained  from  the  Headquarters, 
Airforce  Manpower  and  Personnel  Center.  The  other  data  were  obtained 
from  information  supplied  by  Tactical  Air  Command  Headquarters. 

Table  4  presents  the  results  of  a  cumulative  run,  i.  e.,  one  run, 
for  the  entire  period,  October  1981  through  May  1982,  using  all  4  outputs 
and  8  inputs  defined  in  Table  3.  In  Table  5  the  grouping  is  by  Type  of 
Mission  v/hereas  the  grouping  in  Table  4  is  by  NAF  {  =  Numbered  Air  Force) 
units. 

Appraisal  of  the  units  achieving  100%  efficiency  needs  to  be  held  in 
abeyance  until  some  of  the  qualifications  noted  below  are  attended  to. 

Also,  for  the  present,  attention  in  these  tables  is  confined  only  to  the 
overall  "global"  efficiency  ratings  assigned  to  the  maintenance  operations 
of  each  of  these  wings  by  DEA. 

With  these  provisos  we  may  observe  that  TFW  K  (TFW  =  Tactical  Fighter 
Wing),  flying  old  and  very  complex  aircraft,  has  the  lowest  efficiency 
rating.  See  No.  11  in  Table  4.  Turning  to  Table  5,  it  is  seen  that  this 
0.63  efficiency  rating  is  substantially  below  the  other  DMUs  which  are  also 
engaged  in  both  Operations  and  Training  missions.  On  the  other  hand,  none 
of  the  DMUs  which  have  only  Operations  missions  and  none  of  the  DMUs  which 
have  only  Training  missions  are  flying  the  kins  of  aircraft  being  used  by 
the  3  wings  which  have  both  Operations  and  Training  missions.  Finally,  the 
100%  efficiency  rating  assigned  to  TFW  0  in  NAF  II,  which  also  flies  old 
and  very  complex  aircraft,  is  subject  to  qualification  for  reasons  that  are 
noted  below. 

y  See  also  the  Appendix  where  the  possibility  of  "self  evaluation"  is 
discussed  along  with  ways  in  which  its  occurrence  may  be  identified. 
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NAF 


II 


TABLE  4 

CUMULATIVE  RUN  BY  NAF 
4  OUTPUTS,  8  INPUTS 

DMU,  MISSION.  AIRCRAFT 
TFW  A,  OPS,  New  &  Complex 
TFW  B,  OPS,  New  &  Complex 
TFW  C,  OPS,  Old  &  Complex 
TFW  D,  OPS,  Old  &  Complex 
TFW  E,  OPS,  New  &  Simple 
TFW  F,  OPS  &  TNG,  Very  Old  &  Complex 
TFW  G,  TNG,  New  &  Complex 
V_TFW  H,  TNG,  New  &  Complex 

TFW  I,  OPS,  New  &  Complex 
TFW  J,  OPS  &  TNG,  Old  &  Very  Complex 
TFW  K,  OPS  &  TNG,  Old  &  Very  Complex 
TFW  L,  TNG,  New  &  Complex 
TFW  M,  TNG,  Old  &  Complex 
i^TFW  N,  TNG,  New  &  Simple 


^-4 


EFFICIENCY 

1.00000 

.89355 

.89377 

,90877 

1.00000 

.86043 

,99581 

1.00000 

.99731 

1.00000 

.62381 

1.00000 

1.00000 

1.00000 


’•  *V  *'  - 


TABLE  5 

CUMULATIVE  RUN  BY  TYPE  OF  MISSION 
4  OUTPUTS,  8  INPUTS 


OPS  &  TNG 


DMU,  AIRCRAFT 

EFFICIENCY 

TFW  A,  New  &  Complex 

1.00000 

TFW  B,  New  &  Complex 

.89355 

TFW  Ij  New  &  Complex 

1.00000 

TFW  C,  Old  &  Complex 

.89377 

TFW  D,  Old  &  Complex 

.90877 

TFW  E,  New  &  Simple 

1.00000 

TFW  F,  Very  Old  &  Complex 

.86043 

TFW  J,  Old  &  Very  Complex 

1.00000 

TFW  K,  Old  &  Very  Complex 

.62381 

TFW  G,  New  &  Complex 

.99581 

TFW  M,  Old  &  Complex 

1.00000 

TFW  H,  New  &  Complex 

1.00000 

TFW  L,  New  &  Complex 

1.00000 

JFW  N,  New  &  Complex 

1.00000 
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6.  WINDOW  ANALYSES 

As  noted  earlier  in  this  report,  the  optimization  used  in  DEa  differs 
from  the  ones  used  in  statistics,  e.  g.,  in  least  squares  regression  and 
correlation  analyses.  The  latter  are  directed  to  the  optimizations  across 
all  observations  so  that  the  results  are  applicable  to  the  "class" 
properties  of  all  of  these  observations  rather  than  to  any  one  of  them  in 
particular.  In  DEA  an  opposite  orientation  is  used.  The  optimizations 
are  undertaken  with  reference  to  each  observation  as,  one  after  another, 
the  different  DMUs  are  positioned  in  the  objective  of  (1)  for  evaluation. 

In  a  real  sense,  then,  a  DEA  analysis  should  be  tested  for  its  results 
by  reference  to  the  behavior  of  each  DMU  that  may  be  of  interest  within  the 
observation  set.  Nevertheless,  any  DEA  analysis  can,  and  should,  be  tested 
in  a  variety  of  ways  besides  the  ones  admitted  by  direct  observation.  The 
mathematical  optimizations  and  the  structural  properties  of  a  DEA  model  can 
be  drawn  on  for  these  purposes. 

The  "window  analyses"—^  of  Tables  6  and  7  may  be  used  for  illustration. 
The  results  reported  in  the  preceding  Tables  4  and  5  involved  4  inputs  and  8 
outputs  with  all  values  cumulated  acrQss  the  entire  period  for  use  in 
evaluating  each  of  14  DMUs.  This  comes  close  to  pressing  the  limits  of  the 
degrees -of-freedom  requirements  for  effecting  such  efficiency  evaluations 
in  DEA--or  any  other  similar  evaluation  such  as,  e.  g.,  those  of  a 
statistical  variety.  Hence  it  is  desirable  to  employ  methods  which  introduce 
more  "degrees  of  freedom"  into  the  analysis  and  which  here,  co-incidentally, 
can  also  serve  to  check  other  results. 


y  These  "window  analyses"  were  first  employed  in  [7]  for  DEA  evaluations 
of  Army  recruitment  district  offices. 
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Table  6,  the  Window  Analysis  for  the  wings  in  NAF  I,  provides  an 
illustration.  Each  wing  is  represented  as  if  it  were  a  different  DMU  for 
each  of  3  successive  months.  Then  a  new  3-month  set,  or  "window,"  is 
erected  in  a  similar  manner  starting  with  the  second  month  and  the  process 
then  continues  for  the  results  portrayed  in  Table  6. 

We  illustrate  as  follows:  For  the  first  "window,"  TFW  A  is  represented 
in  the  constraints  of  (1)  as  though  it  were  a  different  DMU  in  October  81, 
November  81  and  December  81.  Therefore  when  TFW  A  is  to  be  evaluated  for 
its  October  81  efficiency,  its  own  performance  data  for  November  81  and 
December  81  for  TFW  A  are  also  included  in  the  constraint  sets  along  with 
a  similar  3-period  representation  of  the  other  TFWs  in  NAF  I.  The  efficiency 
values  of  97.89  and  97.31  as  obtained  in  this  manner  thus  apply  to  TFW  A 
for  October  81  and  November  81,  respectively,  with  the  value  of  h.*  =  98.14 
resulting  from  a  similar  insertion  of  the  December  81  behavior  into  the 
objective  of  (1)  for  evaluation.  To  recapitulate,  the  efficiency  of  TFW  A 
in  each  of  these  months  is  thereby  rated  relative  to  its  own  performance 
as  well  as  the  performance  of  the  other  wings  in  NAF  I  in  each  of  these 
three  months.  This,  in  effect,  provides  a  3-fold  increase  in  the  number 
of  DMUs  considered  in  these  evaluations. 

Notice  that  the  successive  overlapping  windows  provide  a  means  of 
assessing  the  temporal  behavior  of  the  wings.  This  can  be  ef  importance 
in  its  own  right,  as  when,  e.  g.,  in  the  case  of  Army  recruitment 
evaluations  it  was  desirable  to  examine  the  behavior  of  recruitment 
districts  in  different  quarters  of  the  year  to  allow  for  substantial 
seasonal  variations.  The  overlapping  windows  also  provide  a  basis  for 
evaluating  the  stability  of  the  efficiency  ratings  achieved  by  the  DMUs 
(  -  TFWs)  when  they  are  obtained  from  different  data  sets. 
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TABLE  6 

"WINDOW"  ANALYSIS  BY  MONTHS  OF  WINGS’  EFFICIENCY:  NAF  I 


WING 

OCT  81 

NOV  81 

DEC  81 

JAN  82 

MAR  82 

APR  82 

MAY  82 

TFW  A 

97.89 

97.31 

98.14 

97.36 

97.53 

97.04 

96.21 

95.92 

94.54 

95.79 

94.63 

97.64 

94.33 

97.24 

97.74 

TFW  B 

93.90 

95.67 

96.14 

96.72 

96.42 

94.63 

95.75 

94.14 

93.26 

94.54 

93.46 

96.02 

93.02 

96.02 

94.49 

TFW  C 

93.77 

91.53 

95.26 

91.77 

95.55 

94.29 

93.21 

95.04 

94.83 

93.20 

93.09 

92.21 

93.59. 

92.32 

92.83 

TFW  D 

99.72 

96.15 

95.06 

97.91 

95.70 

1.000 

94.79 

1.000 

94.51 

99.71 

94.39 

94.76 

94.95 

94.67 

89.37 

TFW  E 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

98.97 

99.05 

1.000 

99.37 

1.000 

1.000 

1.000 

1.000 

1.000 

TFW  F 

97.42 

93.48 

96.07 

93. CO 

96.24 

93.56 

94.46 

91.75 

92.49 

91.73 

92.32 

92.35 

92.68 

91.98 

99.64 

TFW  G 

90.98 

92.80 

95.96 

93.67 

96.80 

99.52 

93.34 

94.48* 

91.73 

91.94 

89.79 

95.58 

89.35 

95,14 

96.38 

TFW  H 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

*Extretne  mismatch  (over  0.05  change) 
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Consider,  for  instance,  the  second  line  in  Table  6  where  still  another 
value  of  h^*  =  97.36  appears  for  the  November  81  efficiency  rating  of  TFW  A. 
This  rating  emerges  from  yet  another  set  of  3  x  14  =  52  constraining  relations 
used  in  the  evaluation  of  TFW  A  when  it  is  placed  in  the  objective  of  (1)  and 
related  to  the  data  of  November  81,  December  81  and  January  82  for  each  of 
the  TFW  in  NAF  I,  Evidently  the  value  of  97.36  does  not  differ  very  much 
from  the  97.31  efficiency  value  secured  from  the  preceding  set  for  the 
November  81  behavior  of  TFW  A  of  NAF  I.  Indeed,  with  these  "same  month" 
comparisons  only  1  pair  of  values,  as  show  for  TFW  6  in  January  82,  differs 
by  more  than  5%  so  that,  in  general,  these  efficiency  ratings  exhibit  stable 
behavior. 

Turning  to  Table  7  we  are  now  in  a  position  to  say  something  more  about 
the  efficiency  evaluations  of  TFWs  J  and  K  which  attracted  our  attention  in 
Table  5.  Although  never  falling  to  the  very  low  value  of  h^*  =  .63  to  which 
it  accumulated  in  Table  5,  the  efficiency  ratings  for  wing  K  are  generally 
low  and  declining.  The  one  reversal,  93.74,  which  is  starred  in  January  82 
is  really  very  far  out  of  line  with  the  other  values  in  this  set  and  needs 
to  be  marked  for  further  investigation  as  to  possible  mis-reporting  or  to 
some  error  in  the  data  or  the  computations. 

Turning  to  the  efficiency  evaluations  for  TFW  J  in  Table  7,  we  find 
that  we  have  additional  reasons  to  believe  that  the  100%  efficiency  rating 
exhibited  for  this  DMU  in  Tables  4  and  5  is  likely  to  have  resulted  from 
special  {and  possibly  misleading)  features  in  the  analyses  underlying  those 
Tables.  Note  that  the  efficiency  ratings  of  TFW  J  are  persistently  below 
those  of  all  of  the  other  wings  except  for  wing  K.  In  contrast  to  the  latter, 
however,  the  efficiency  ratings  of  TFW  J  do  exhibit  an  upward  trend, 
especially  in  the  later  parts  of  the  period  and  this,  too,  is  information 
of  potential  use  in  evaluating  its  behavior. 


TABLE  7 


"WINDOW" 

ANALYSIS 

BY  MONTHS  OF  WINGS' 

EFFICIENCY 

;  NAF  II 

WING 

OCT  81 

NOV  81 

DEC  81 

JAN  82 

MAR  82 

APR  82 

MAY  82 

TFW  I 

99.11 

95.94 

96.04 

99.76 

1.000 

98.16 

1.000 

98.99 

98.97 

94.59 

94.62 

94.68 

99.16 

98.92 

97.28 

TFW  J 

92.85 

90.90 

91.50 

91.62 

92.12 

90.26 

94.75 

93.39 

92.92 

93.83 

93.84 
94.52 

95.33 

96.07 

94.43 

TFW  K 

86.25 

84.42 

84.98 

84.03 

84.47 

83.37 

93.74* 

82.54 

82.39 

80.26 

80.14 

80.96 

79.58 

78.66 

79.75 

TFW  L 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

99.55 

99.39 

1.000 

97.39 

96.85 

96.66 

1.000 

1.000 

1.000 

TFW  M 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

i.ooo 

1.000 

1.000 

1.000 

1.000 

1.000 

98.75 

98,51 

99.59 

TFW  N 

i  .UUU 

^  AAA 

i  .  uuu 
1.000 

96.63 

1.000 

99.45 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

*Extreme  mismatch  (over  0.05  change) 


Table  8,  below,  summarizes  the  results  in  Tables  6  and  7  in  yet  another 
way.  Note,  for  example,  that  TFW  K  has  not  only  the  lowest  mean  but  it  also 
has  the  highest  variance  in  its  efficiency  ratings.  Part  of  the  latter  may 
be  due  to  the  unusually  high  rating  of  93.74  in  the  January  82  evaluation 
of  this  DMU  which,  as  we  previously  observed,  should  be  set  aside  for  further 
examination.  On  the  other  hand,  low  means  tend  to  be  accompanied  by  high 
variances  in  Table  8,  with  the  possible  exception  of  TFW  D.  The  latter, 
which  also  appears  in  group  D  for  Table  8,  however,  does  have  a  marked  downturn 
in  its  efficiency  ratings  starting  in  March  82--see  Table  6--and  this,  too, 
should  be  investigated  in  more  detail. 

8.  EXAMPLE  SOLUTION 

Table  9,  below,  will  help  to  interpret  Table  5  and  it  will  also  provide 
additional  information  in  its  own  right.  It  provides  part  of  a  computer 
printout  which  includes  the  h^*  =  0.86  value  for  DMU  7  =  TFW  F  that  was 
reported  in  the  cumulative  evaluation  used  for  Table  5. 

Turning  first  to  the  outputs  we  note  that  the  "value  if  efficient"  is 
obtained  by  simply  adding  the  "slack"  value  to  the  "value  measured" 

(  =  reported  or  observed  value).  E.  g.,  the  8,127  for  CUMSORTFLO 
(  =  Cumulative  Sorties  Flown)  is  obtained  by  adding  the  314,  under  "slack", 
to  the  7:813  under  "value  measured." 

The  other  output  "values  if  efficient"  recorded  in  the  last  column  of 
Table  9  are  obtained  in  the  same  way  with,  of  course,  those  having  zero 
slack  requiring  no  such  addition  to  achieve  their  "value  if  efficient." 

The  input  "values  if  efficient"  are  obtained  in  a  somewhat  more 

* 

elaborate  way  by  first  applying  the  efficiency  value  of  h^  =  0.86  to  the 
value  measured  and  then  subtracting  the  slack.  The  basic  idea  can  be 
illustrated  via  the  NOCHARGl.OS  (  =  Number  of  sortie  losses  due  to  external 


TABLE  8 

MEAN-VARIANCE  ANALYSIS  ACROSS  TIME 


WING 

MEAN 

TFW  A 

96.62 

TFW  B 

94.95 

TFW  C 

93.50 

TFW  0 

96.11 

TFW  E 

99.89 

TFW  F 

93.98 

TFW  G 

93.87 

TFW  H 

100.00 

TFW  I 

97.75 

TFW  J 

93.22 

TFW  K 

83.04 

TFW  L 

99.32 

TFW  M 

99.79 

TFW  N 

99.96 

Groups:  A  -  very  low 
B  -  low 

C  -  medium  size 
D  -  high  a* 


VARIANCE 
1.567 
1.419 
1.523 
7.913 
0.085 
5.167 
7.400 
0.000 
3.880 
2.664 
13.114 
1.439 
0.219 
0.019 
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TABLE  9 

SOLUTION  FOR  DMU  7*  =  TFW  F 
EFFICIENCY  -  0.86 

facet  =11,  8 


OUTPUTS: 

VALUE 

MEASURED 

SLACK 

VALUE  IF 
EFFICIENT 

CUMSORTFLO 

7,813 

314 

8,127 

26,225 

MICAPHOURS 

26,225 

0 

TNMCHM 

1,380 

1,275 

2,655 

FIXRATE 

4,786 

0 

4,786 

INPUTS: 

NOCHARGLOS 

9,841 

2,000 

6,463 

5,113 

CUMPOSSAIR 

5.945 

0 

TNMCHS 

4,800 

1,915 

2,213 

1 ,341 

OFASSIGNED 

1,900 

293 

OFMEASURE 

1,117 

255 

705 

AIRMENASSI 

1,376 

279 

903 

AIRMENMEAS 

1,097 

223 

720 

CANRATE 

9,382 

1,546 

6,522 

*  The  DMU  number  is  keyed  to  Table  5. 


-25- 


reasons  like  weather)  as  follows;  0.86  x  9,841  -  2,000  =  6,463.  In  other 
words,  the  value  if  efficient  is  obtained  by  first  reducing  this  input  to 
86%  of  its  observed  value  and  then  subtracting  slack.  This  is  to  say  that 
TFW  F  should  not  have  required  any  more  than  86%  of  the  reported 
input  levels.  For  all  inputs  except  CUMPOSSAIR  (  =  the  cumulative  value 
of  TOTPOSSAIR),  this  wing  should  also  have  required  inputs  in  the  still 
further  reduced  amounts  indicated  in  the  slack  column. 

Here  we  have  brought  the  variable  NOCHARGLOS  up  front  in  order  to 
observe  that  some  of  these  values  are  in  need  of  qualified  interpretations 
and  possibly  further  treatment  as  well.  In  all  cases  these  computer 
printouts  such  as  the  one  in  Table  9  will  need  to  be  buttressed  by  competent 
judgment  aided,  perhaps,  by  addition-'.!  inquiries  directed  to  each  individual 
DMU.  In  the  portrayal  used  here,  moreover,  each  input  is  treated  as  though 
it  can  be  varied  at  niariagenierit  discretion.  This  is  not  true  for  weather 
dependent  variables  like  NOCHARGLOS,  however,  and  so  further  treatment  is 
needed  for  this  input  as  being  a  "non-discretionary  resource,"  at  least  in 
part. 

It  is  not  proposed  to  undertake  a  discussion  here  of  the  details  on 
how  such  adjustments  can  be  made  for  non-discretionary  inputs.  The  point 
does  need  to  be  made  explicitly  that  such  non-discretionary  inputs— e.  g., 
favorable  flying  weather^-can  be  important  and  when  this  is  the  case 
they  should  not  be  omitted.  Judged  from  the  standpoint  of  output  attain¬ 
ment,  the  evaluation  should  take  account  of  these  inputs  even  when  non- 
discretionary.  In  a  DEA  analysis,  this  is  done  by  reference  to  relative 

y  Examples  of  non-discretionary  inputs  in  these  studies  have  included 
unemployment  rates  in  Army  recruitment  districts  and  parental  and 
neighborhood  backgrounds  as  inputs  to  educational  attainment  by 
school  children.  See  [7]  and  [14]. 
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performances  achieved  by  other  units  such  as  those  identified  as  facet 
members  for  DMU  7  at  the  top  of  Table  9  which  also  utilize  these  kinds  of 
non-discretionary  inputs. 

Before  turning  to  a  discussion  of  these  facet  members,  we  should 
emphasize  that  full  efficiency  is  achieved  in  a  DEA  analysis  only  when 
all  of  the  indicated  adjustments  can  be  accomplished.  That  is,  all  output 
and  input  adjustments,  as  in  the  final  column  of  Table  9,  are  needed  to 
achieve  full  efficiency.  Thus,  to  rate  as  100%  efficient,  TFW  F 
would  have  had  to  increase  its  outputs  and  reduce  its  inputs  and  this, 
in  DEA  terminology,  would  have  projected  it  on  to  its  efficient  facet. 

9.  FACET  MEMBERS 

The  efficiency  rating  is  only  relative,  as  was  noted  at  the  outset 
of  this  paper.  Here  the  rating  is  determined  relative  to  DMUs  11  and  8 
which,  as  can  be  seen  in  Table  5,  are  both  rated  as  100%  efficient  via 
the  cumulative  runs  which  form  the  basis  of  the  evaluation  in  Table  9. 

All  evaluations  in  a  DEA  analysis  are  effected  by  reference  to 
subsets  of  DMUs  which  are  rated  as  100%  efficient.  Among  such  efficient 
subsets  of  wings,  the  optimization  in  (1)  has  picked  DMUs  11  and  8  as 
the  "best"  reference  set  for  DMU  7  =  TFW  F  in  this  case. 

Formally,  no  other  reference  set  can  yield  a  higher  h^*  value  for 
TFW  F,  since  otherwise  the  0.86  efficiency  value  would  not  be  maximal. 
Finally,  but  less  formally,  we  may  say  that  each  DEA  optimization  tends 
to  pick  the  reference  sets  for  these  evaluations  from  among  the  available 
(efficient)  DMUs  which  are  most  like  the  unit  being  evaluated.  Thus  wing  F 
which  has  the  mission  of  both  training  and  operations,  while  flying  very 
old  and  complex  aircraft,  is  evaluated  by  reference  to  DMU  11,  which  is 
TFW  M  with  training  as  its  mission  while  flying  old  and  complex  aircraft. 
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and  by  reference  to  DMU  8,  which  is  TFW  J  with  the  mission  of  both 
training  and  operations  while  flying  old  and  very  complex  aircraft. 

One  reason  for  displaying  the  OMUs  in  the  facet  set,  as  is  done 
in  Table  9,  should  now  be  apparent  since  this  information  supplies 
further  insight  into  the  standpoint  from  which  the  evaluation  was 
effected.  Hence,  this  information,  too,  can  be  taken  into  account  in 
any  managerial  follow-ups  that  might  be  effected  on  the  basis  of  such 
DEA-initiated  inquiries.  In  particular  this  facet  evaluation  information 
suggests  that  these  members  of  the  "efficient"  DMUs  should  provide 
information  for  study  in  the  evaluation  of  the  inefficient  DMUs  which 
they  help  to  rate. 

There  is  a  great  deal  of  additional  information  for  managerial 
use  that  is  included  in  the  computer  printouts  that  are  availa  le  from 
such  DEA  studies.  Now,  however,  we  want  to  turn  attention  to  Table  10 
which  records  the  number  of  appearances  of  each  wing  as  a  member  of  a 
facet  used  to  evaluate  other  wings.  Here  the  reference  is  to  the 
efficiency  evaluations  reported  in  the  window  analyses  of  Tables  6  and  7. 

Note,  first,  that  TFW  G  does  not  appear  in  any  of  the  facets  used  to 
evaluate  other  wings.  This  provides  further  evidence  that  this  wing  is 
not  really  efficient  but  is  being  rated  so--perhaps  justifiably--by  virtue 
of  its  specialized  features.  In  contrast  TFW  E  appears  189  times  as  a 
member  of  an  efficient  facet--i.e.,  as  a  member  of  the  efficient  reference 
group--used  to  evaluate  other  DMUs,  and  this  is  corroborating  evidence  of 
its  relative  efficiency.  Notice,  for  instance,  that  this  performance 
exceeds  event  that  of  TFW  H  despite  the  fact  that  the  latter  achieved  a 
rating  of  100%  efficiency  in  every  one  of  the  evaluations  reported  in 
Table  7. 
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The  important  point  to  make  here  is  that  such  corroboration  is 
available  and  can  be  effected  in  a  variety  of  other  ways  in  DEA  analyses. 
Finally,  and  equally  important,  this  kind  of  information  can  be  used  in  a 
variety  of  additional  ways  that  can  have  managerial  value.  For  instance, 
Table  10  can  be  used  to  guide  analyses  directed  to  ascertaining  the  effect 
on  other  DMUs  which  might  accrue  from  altering  the  behavior  of  any  of  the 
particular  DMUs  in  that  Table.  Notice,  for  instance,  that  a  reorganization 
which  eliminated  the  TFW  E  could  affect  the  efficiency  evaluations  of 
many  other  DMUs  whereas  a  similar  reorganization  of  the  TFW  G  would 
not  be  likely  to  have  any  such  further  consequences.—^ 

10.  CONCLUSION 

The  above  discussion  does  not  exhaust  the  uses  of  DEA.  We  have 
not  touched  on  the  duality  relations,  for  instance,  and  how  the 
information  that  is  generated  from  these  aspects  of  the  optimizations 
can  be  put  to  managerial  use.  All  that  needs  to  be  said,  at  this  point, 
is  that  the  situation  is  analogous  to  the  ones  found  in  other  parts  of 
mathematical  programming  where  these  duality  relations  have  added  major 
features  to  the  "policy  analyses"  that  have  accompanied  mathematical 
programming  applications  in  many  different  types  of  management  planning 
problems.-'^  Here  the  values  of  these  dual  variables  provide  important 
information  on  possible  substitutions  and  tradeoffs  that  can  be  used  to 
further  other  managerial  objectives  after  efficiency  has  been  achieved 
by  each  DMU. 

1/  See  [2]  for  further  discussion  in  the  context  of  reorganizations  proposed 
for  the  San  Antonio  Conmunity  College. 

2/  See  [5]  or  [3]  for  detailed  examples  and  discussions. 


TABLE  10 


NO.  OF  APPEARANCES  OF  EACH  WING  IN  THE  FACET  OF  ANOTHER  WING* 


\WINGS  IN 

wingNrcet 

D 

G 

H 

E 

I 

L 

M 

N 

A 

1 

13 

19 

3 

13 

9 

5 

B 

14 

17 

2 

7 

15 

4 

C 

5 

1 

16 

2 

15 

13 

D 

® 

5 

17 

2 

19 

9 

G 

® 

2 

24 

5 

2 

8 

H 

24 

11 

1 

9 

7 

F 

4 

3 

16 

3 

17 

7 

E 

8  i 

20 

5 

5 

5 

1 

6 

18 

0 

7 

15 

9 

L 

10 

7 

M 

7 

M 

1 

6 

8 

1 

Hi 

6 

J 

3 

16 

17 

12 

K 

3 

4 

17 

1 

15 

7 

N 

2 

3 

1 

TOTAL  NO.  OF 
APPEARANCES  IN 
OTHER  FACETS 

17 

0 

74  189 

5 

48 

138 

99 

*Maximutn  possible  =  45  facet  appearances  =  5  windows  x  3  DMU  repetitions  per 

window  X  3  possible  appearances  in  a  facet. 
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"Tradeoffs as  the  name  itself  suggests,  are  not  required  until 
after  efficiency  has  been  achieved  by  each  DMU.  See  the  definitions  of 
efficiency  contained  in  section  1,  p.  2,  above.  In  other  words,  the 
tradeoff  concept  is  related  to  the  notion  of  an  efficiency  frontier  within 
which  improvements  in  any  input  or  output  can  be  achieved  only  at  the 
expense  of  other  inputs  and  outputs.  To  guide  decisions  where  some  out¬ 
puts  are  to  be  improved  at  the  expense  of  others,  it  is  important  to  know 
the  tradeoff  (or  substitution)  ratios  as  well  as  the  ranges  of  input  and 
output  alterations  within  which  these  ratios  are  valid.  Much  of  this  is 
supplied  automatically  in  the  computer  printouts.  Further  research  under 
way  is  directed  toward  providing  an  even  more  complete  picture. 

Via  routes  like  these  one  is  able  to  move  from  "control"  and 

evaluation  of  each  DMU  to  other  (related)  problems  such  as  "planning"-^ 
for  still  more  effective  performance  by  resource  allocations  across 

DMUs  by  methods  such  as  "goal  programming,"  etc.,  which  the  authors  of 

this  report  have  developed  for  similar  uses  in  other  military  planning 
2/ 

contexts  .- 

In  this  report  we  have  confined  ourselves  to  straightforward  uses 
of  what  is  already  available  from  DEA.  The  preceding  examples  are  only 
il lustrative--for  use  in  examining  DEA—rather  than  directed  toward 
actual  managerial  evaluation  of  performance  of  the  air  force  wings 
included  in  these  illustrative  examples. 

Within  this  context  we  have  also  tried  to  highlight  differences 
between  DEA  and  other  approaches  to  the  topics  examined  here.  This  has 
included  the  comments  we  offered  in  section  1  on  possible  uses  of  ratio 

y  The  distinctions  between  planning  and  control  uses  of  mathematical 
prog ratrani rig  are  elaborated  in  further  detail  in  Chapter  1  of  [5]. 

2/  See,  e.g.,  [4]  and  [13]. 
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analysiSj  regression  and  simulation  techniques  which  also  have  been  used 
in  past  and  present  efficiency  evaluations.  A  use  of  DEA  does  not, 
however,  preclude  the  use  of  these  other  techniques.  On  the  contrary, 

DEA  is  best  regarded  as  an  additional  method  of  analysis  which  can  be 
used  in  conjunction  with  any  or  all  of  these  other  presently  used 

alternatives.  This  should  therefore  be  born  in  mind  when  considering 
the  following  recapitulation. 

In  contrast  with  the  usual  ratio  approaches,  DEA  deals  simultaneously 
with  all  inputs  and  outputs  that  are  considered  to  be  pertinent,  and  in 
whatever  mixes  are  present,  in  the  performance  reports  of  a1 1  DMUs  when 
arriving  at  its  efficiency  evaluation  ratios.  See  expression  (2.1)  to 
(2.4)  and  the  accompanying  discussions  on  page  7  in  section  2,  above. 

In  contrast  with  the  usual  regression  approaches--e.  g.,  as  in 
standard  versions  of  least  squares-correlation  analyses— DEA  optimizes 
on  the  observations  for  each  DMU.  Moreover,  instead  of  being  directed 
toward  averages,  the  focus  on  DEA  is  directed  to  frontier  location  and 
estimation.  These  tv/in  attributes  then  lend  themselves  to  the  kinds  of 
inefficiency  estimates  and  characterizations  for  each  DMU  that  were 
discussed  in  connection  with  Table  10  at  the  end  of  the  preceding  section. 

In  contrast  with  the  usual  simulation  modeling  approaches,  DEA  does 
involve  optimizations  and  via  this  route  it  provides  access  to  bodies  of 
theory  that  are  available  both  for  immediate  use  and  further  extensions. 
Access  to  this  prior  body  of  knowledge  was  used,  for  example,  in  the 
identification  of  the  facet  members  that  were  discussed  as  providing  the 
basis  used  for  the  efficient  facet  evaluation  of  the  (cumulative  run) 
performance  of  the  TFW  f-  in  Table  10.  It  also  provided  guidance  for 
interpreting  the  results  of  the  window  analysis  by  reference  to  the 


number  of  facet  appearances  in  evaluating  other  wings.  In  this  manner 
we  were  able  to  identify  facets  which  had  achieved  100%  efficiency 
ratings  by  means  of  "self"  comparisons--e.  g.,  because  of  one  or  more 
special  features  of  the  operations  of  these  particular  DMUs— as  well 
as  others  who  had  achieved  100%  relative  efficiency  by  virtue  of 
comparisons  with  a  great  many  other  wings. 

In  conclusion  we  may  therefore  point  to  the  window  analysis  exhibited 
in  Tables  6  and  7.  This  window  analysis  is  a  new  feature  of  DEA  which  has 
emerged  from  recent  applications  to  military  problems.-'^  It  seems  likely 
that  still  other  such  developments  will  emerge  from  similar  research  on 
DEA  for  use  on  Air  Force  problems  which,  we  hope,  has  been  demonstrated 
as  having  potential  value  for  use  in  controlling  and  managing  Air  Force 
resources. 


1/  Its  first  appearance  was  in  [7]. 
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APPENDIX 


In  this  section  we  present  the  rigorous  mathematical  form  for  the  Data 
Envelopment  Analysis  of  the  CCR  ratio  form  for  relative  efficiency  and  exhibit 
the  type  of  mathematical  properties,  duality,  etc.  that  the  mathematical 
system  has  by  means  of  a  proof  of  the  efficiency  of  the  projection  of  an 
inefficient  DMU  onto  the  facet  of  its  associated  efficient  DMUs. 

Of  course  this  is  not  the  only  type  of  change  in  inputs  and  outputs  which 
could  bring  the  DMU  to  a  relative  efficiency  of  1,  ceteris  paribus.  For 
example,  if  the  DMU  has  an  efficiency  of  h^,  then  raising  all  its  outputs  by 
a  factor  (l/h^)  would  result  in  an  efficiency  of  unity. 

Other  mathematical  properties  of  the  Data  Envelopment  Analysis  systems 
useful  for  analysis  and  evaluation  of  desirable  directions  for  improvement  of 
particular  desired  outputs  are  to  be  found  in  CCS  459,  "Pareto-Optimal ity, 
Efficiency  Analysis  and  Empirical  Production  Functions,"  by  A,  Charnes,  W.W. 
Cooper,  B.  Golany,  L.  Seiford,  J.  Stutz,  May  1983. 

We  now  move  from  the  individual  variable  form  in  the  text  to  vector-matrix 
notation.  The  CCR  ratio  problem  then  is  expressed  as: 


Max 


subject  to 


^  1 


>j“l»  •  •  • 


(A) 


^  -ee' 


<  -ee' 


with  e  >  0  as  a  non-Archimedean  infinitesimal. 
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Here  the  input  vector  X.  and  output  vector  Y.  are  assumed  to  have  positive 

V  J 

components.  The  quantities  and  ^  are  called  "virtual  multipliers."  The 
scalar  products  n^Y-  ,  C^X.  are  called  "virtual  outputs"  and  "virtual  inputs" 

•J  J 

respectively.  The  inequalities 


are  called  the  "efficiency  technology."  The  further  conditions  on 
are  called  the  "multiplier  positivity"  conditions  since  they  assure  (i)  that 
only  positive  values  are  secured  for  these  and  (ii)  that  there  is  algebraic 
closure  to  the  inequality  system. 

By  means  of  the  Charnes-Cooper  transformation  of  linear  fractional 
T  T  T  T  T  T 

programming  y  =  ^  X^  ,  v  =  C  /Ci  ,  we  obtain  a  linear  programming 
problem  and  its  dual  as  shown  in  (A. 2). 

The  dual  LP  problems  may  be  characterized  respectively  as: 

(1)  Maximize  the  virtual  output  of  DMU^  at  unit  virtual  input 
subject  to  the  efficiency  technology  and  positivity  conditions. 

(2)  Minimize  the  intensity  of  the  input  vector  with  the  input 
and  output  vectors  "enveloped"  respectively  from  below  and  above“-i.e., 

"Data  Envelopment  Analysis." 
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Projection  on  Efficiency  Facets 


The  dual  non-Archimedean  linear  programs  for  evaluating  the  relative 
efficiency  of  the  DMU  with  input-output  vectors  are: 


max  y  Y, 


min  0 


T  +  T  + 
e  e  s  -  c  e  s 


subject  to 
(A.2) 


v'X^  =  1 
o 

y^Y  -  v''^X  <0 


<  -  e  e 

-v^  <  -  €  e^ 


YX  -  s" 


QXq  -  XX 


■s"  =  0 


X  ,  s  ,5“  >0 


where  X  =  [Xj,...,Xj^]  ,  Y  =  [Yj^,...,Y^]  ,  £  is  the  non-Archimedean  infinitesimal 

If  0*  ,  X|  ,  ,  sg~  is  an  optimal  basic  solution  {with  dual  evaluators 

y*  ,  vg)  designating  the  coefficient  vectors  or  matrices 


{A.3) 


as  the  basis  of  this  optimal  solution,  they  satisfy  the  reduced  system  of 
equations: 


(A.4) 


-  '‘b'b 


-S*'  =  0 

5b  U 


If  one  replaces  (X„,Y^)  by  X;  =  9*Xq  -  s*'  and  Y^  =  Y^  +  s*  , 


'  O’  0 


'o  'o  B 


then  0  =  1  ,  X  =  X*  ,  s'  =  0  ,  Sr,  =  0  is  a  feasible  basic  solution  for 

DO  O 

(X^,  Y^)  with  this  same  basis. 
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Now  are  still  the  dual  evaluators  for  the  basis  and 

(^*  5  )  T  T  T  I 

^B  ^B  B  "  ^B  ^  "  ^B  ^0 

=  v*^(®*Xq')  .  where  «  =  1 


But  y*'YB 


Hence 


(A.6) 


'‘b'<0 


=  0  for  the  dual  inequalities  designated  by 


0 


i.e.,  the  new  inequality  replacing  <0  is  also  satisfied  by 

y|^  >  •  Further 


Thus  iig  -  \1q 


/(••'■;) 


^B  =  ^B 


'/(-'■;) 


is  a  feasible  (basic) 


solution  to  the  (X^jY^)  problem  with  functional  value  Pb  =  1  equal  to 


the  dual  problem  functional  value  9=1.  Thus  the  "projection"  X^  ->  =  0*X^  -  s* 

Y  Y'  =  i  +  s*^  yields  an  efficient  DMU. 

0  0  0 
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